@poppinss/utils

What is @poppinss/utils?

@poppinss/utils is a utility library that provides a collection of helper functions for various common tasks in JavaScript and TypeScript. It includes utilities for data manipulation, type checking, and more.

What are @poppinss/utils's main functionalities?

String Manipulation

The string manipulation utilities include functions like camelCase, which converts a string to camel case.

const { string } = require('@poppinss/utils');

const camelCaseString = string.camelCase('hello world');
console.log(camelCaseString); // 'helloWorld'

Object Utilities

Object utilities include functions like clone, which creates a deep copy of an object.

const { object } = require('@poppinss/utils');

const obj = { a: 1, b: 2 };
const clonedObj = object.clone(obj);
console.log(clonedObj); // { a: 1, b: 2 }

Type Checking

Type checking utilities include functions like is.number and is.string, which check the type of a given value.

const { is } = require('@poppinss/utils');

console.log(is.number(123)); // true
console.log(is.string(123)); // false

Other packages similar to @poppinss/utils

lodash

Lodash is a popular utility library that provides a wide range of functions for data manipulation, type checking, and more. It is more comprehensive and widely used compared to @poppinss/utils.

underscore

Underscore is another utility library similar to Lodash, offering a variety of functions for common programming tasks. It is older and less feature-rich than Lodash but still widely used.

ramda

Ramda is a functional programming library for JavaScript that provides utility functions with a focus on immutability and pure functions. It offers a different approach compared to @poppinss/utils, emphasizing functional programming paradigms.

README

Utils

Collection of reusable scripts used by AdonisJS core team

This module exports a collection of re-usable utilties to avoid re-writing the same code in every other package. We also include a handful of Lodash utilities, which are used across the AdonisJS packages eco-system.

Table of contents

• Installation
• Exception
• fsReadAll
• requireAll
• esmRequire
• esmResolver
• resolveFrom
• resolveDir
• interpolate
• Lodash utilities
  • Exported methods
• Base 64 Encode/Decode
  • encode
  • decode
  • urlEncode
  • urlDecode
• Random String
• Safe equal
• Safe stringify
• Safe parse
• Message Builder
• defineStaticProperty
• compose
  • Mixins gotchas

Installation

Install the package from npm registry as follows:

npm i @poppinss/utils

# yarn
yarn add @poppinss/utils

and then use it as follows:

import { requireAll } from '@poppinss/utils'
requireAll(__dirname)

Exception

A custom exception class that extends the Error class to add support for defining status and error codes.

import { Exception } from '@poppinss/utils'

throw new Exception('Something went wrong', 500, 'E_RUNTIME_EXCEPTION')
throw new Exception('Route not found', 404, 'E_ROUTE_NOT_FOUND')

fsReadAll

A utility to recursively read all script files for a given directory. This method is equivalent to readdir + recursive + filter (.js, .json, .ts).

import { fsReadAll } from '@poppinss/utils'

const files = fsReadAll(__dirname) // array of strings

You can also define your custom filter function. The filter function must return true for files to be included.

const files = fsReadAll(__dirname, (file) => {
  return file.endsWith('.foo.js')
})

requireAll

Same as fsReadAll, but instead require the files. Helpful when you want to load all the config files inside a directory on app boot.

import { requireAll } from '@poppinss/utils'
const config = requireAll(join(__dirname, 'config'))

{
  file1: {}, // exported object
  file2: {} // exported object
}

esmRequire

Utility to require script files wihtout worrying about CommonJs and ESM exports. This is how it works.

• Returns the exported value for module.exports.
• Returns the default value is an ESM module has export default.
• Returns all exports if is an ESM module and doesn't have export default.

foo.js

module.exports = {
  greeting: 'Hello world',
}

foo.default.js

export default {
  greeting: 'Hello world',
}

foo.esm.js

export const greeting = {
  greeting: 'hello world',
}

import { esmRequire } from '@poppinss/utils'

esmRequire('./foo.js') // { greeting: 'hello world' }
esmRequire('./foo.default.js') // { greeting: 'hello world' }
esmRequire('./foo.esm.js') // { greeting: { greeting: 'hello world' } }

esmResolver

The esmResolver method works similar to esmRequire. However, instead of requiring the file, it accepts the object and returns the exported as per the same logic defined above.

import { esmRequire } from '@poppinss/utils'

esmResolver({ greeting: 'hello world' }) // { greeting: 'hello world' }

esmResolver({
  default: { greeting: 'hello world' },
  __esModule: true,
}) // { greeting: 'hello world' }

esmResolver({
  greeting: { greeting: 'hello world' },
  __esModule: true,
}) // { greeting: { greeting: 'hello world' } }

resolveFrom

Works similar to require.resolve, however it handles the absolute paths properly.

import { resolveFrom } from '@poppinss/utils'

resolveFrom(__dirname, 'npm-package') // returns path to package "main" file
resolveFrom(__dirname, './foo.js') // returns path to `foo.js` (if exists)
resolveFrom(__dirname, join(__dirname, './foo.js')) // returns path to `foo.js` (if exists)

resolveDir

The require.resolve or resolveFrom method can only resolve paths to a given file and not the directory. For example: If you pass path to a directory, then it will search for index.js inside it and in case of a package, it will be search for main entry point.

On the other hand, the resolveDir method can also resolve path to directories using following resolution.

• Absolute paths are returned as it is.
• Relative paths starting with ./ or .\ are resolved using path.join.
• Path to packages inside node_modules are resolved as follows: - Uses require.resolve to resolve the package.json file. - Then replace the package-name with the absolute resolved package path.

import { resolveDir } from '@poppinss/utils'

resolveDir(__dirname, './database/migrations')
// __dirname + /database/migrations

resolveDir(__dirname, 'some-package/database/migrations')
// {path-to-package}/database/migrations

resolveDir(__dirname, '@some/package/database/migrations')
// {path-to-package}/database/migrations

interpolate

A small utility function to interpolate values inside a string.

import { interpolate } from '@poppinss/utils'

interpolate('hello {{ username }}', { username: 'virk' })
interpolate('hello {{ users.0.username }}', { users: [{ username: 'virk' }] })

If value is missing, it will be replaced with an undefined string.

Lodash utilities

Lodash itself is a bulky library and most of the times, we don't need all the functions from it. For this purpose, the lodash team decided to publish individual methods to npm as packages. However, most of those individual packages are outdated.

At this point, whether we should use the complete lodash build or use outdated individual packages. Both are not acceptable.

Instead, we make use of lodash-cli to create a custom build of all the utilities we ever need inside the AdonisJS eco-system and export it as part of this package. Why part of this package?

Well, creating custom builds in multiple packages will cause friction, so it's better to keep it at a single place.

Do note: There are no Typescript types for the lodash methods, since their CLI doesn't generate one and the one published on @types/lodash package are again maintained by community and not the lodash core team, so at times, they can also be outdated.

import { lodash } from '@poppinss/utils'
lodash.snakeCase('HelloWorld') // hello_world

Exported methods

Following is the list of exported helpers.

• pick
• omit
• get
• set
• unset
• mergeWith
• merge
• snakeCase
• camelCase
• startCase

Base 64 Encode/Decode

Following helpers for base64 encoding/decoding also exists.

encode

import { base64 } from '@poppinss/utils'
base64.encode('hello world')
base64.encode(Buffer.from('hello world', 'binary'))

decode

import { base64 } from '@poppinss/utils'
base64.decode(base64.encode('hello world'))
base64.decode(base64.encode(Buffer.from('hello world', 'binary')), 'binary')

urlEncode

Same as encode, but safe for URLS and Filenames

urlDecode

Same as decode, but decodes the urlEncode output values

Random String

A helper to generate random strings of a given length. Uses crypto under the hood.

import { randomString } from '@poppinss/utils'
randomString(32)
randomString(128)

Safe equal

Compares two values by avoid timing attack. Accepts any input that can be passed to Buffer.from

import { safeValue } from '@poppinss/utils'
if (safeValue('foo', 'foo')) {
}

Safe stringify

Similar to JSON.stringify, but also handles Circular references by removing them.

import { safeStringify } from '@poppinss/utils'

const o = { b: 1, a: 0 }
o.o = o

console.log(safeStringify(o))
// { "b":1,"a":0 }

console.log(JSON.stringify(o))
// TypeError: Converting circular structure to JSON

Safe parse

Similar to JSON.parse, but protects against Prototype Poisoning

import { safeParse } from '@poppinss/utils'

const input = '{ "user": { "__proto__": { "isAdmin": true } } }'

JSON.parse(input)
// { user: { __proto__: { isAdmin: true } } }

safeParse(input)
// { user: {} }

Message Builder

Message builder provides a sane API for stringifying objects similar to JSON.stringify but has a few advantages.

• It is safe from JSON poisoning vulnerability.
• You can define expiry and purpose for the encoding. The verify method will respect these values.

The message builder alone may seem useless, since anyone can decode the object and change its expiry or purpose. However, you can generate an hash of the stringified object and verify for tampering by validating the hash. This is what AdonisJS does for cookies.

import { MessageBuilder } from '@poppinss/utils'
const builder = new MessageBuilder()
const encoded = builder.build({ username: 'virk' }, '1 hour', 'login')

Now verify it

builder.verify(encoded) // returns null, no purpose defined
builder.verify(encoded, 'register') // returns null, purpose mismatch.
builder.verify(encoded, 'login') // return { username: 'virk' }

defineStaticProperty

Explicitly define static properties on a class by checking for hasOwnProperty. In case of inheritance, the properties from the parent class are cloned vs following the prototypal inheritance.

We use/need this copy from parent class behavior a lot in AdonisJS. Here's an example of Lucid models

You create an application wide base model

class AppModel extends BaseModel {
  @column.datetime()
  public createdAt: DateTime
}

AdonisJS will create the $columnDefinitions property on the AppModel class, that holds all the columns

AppModel.$columnDefinitions // { createdAt: { columName: created_at } }

Now, lets create another model inheriting the AppModel

class User extends AppModel {
  @column()
  public id: number
}

As per the Javascript prototypal inheritance. The User model will not contain the columns from the AppModel, because we just re-defined the $columnDefinitions property. However, we don't want this behavior and instead want to copy the columns from the AppModel and then add new columns to it.

Voila! Use the defineStaticProperty helper from this class.

class LucidBaseModel {
  static boot() {
    defineStaticProperty(this, LucidBaseModel, {
      propertyName: '$columnDefinitions',
      defaultValue: {},
      strategy: 'inherit',
    })
  }
}

The defineStaticProperty takes a total of three arguments.

• The first argument is always this.
• The second argument is the root level base class. This will usually be the class exported by your package or module.
• The third argument takes the propertyName, defaultValue (in case, there is nothing to copy), and the strategy.
• The inherit strategy will copy the properties from the base class.
• The define strategy will always use the defaultValue to define the property on the class. In other words, there is no copy behavior, but prototypal inheritance chain is also breaked by explicitly re-defining the property.

compose

Javascript doesn't have a concept of inherting multiple classes together and neither does Typescript. However, the official documentation of Typescript does talks about the concept of mixins.

As per the Typescript docs, you can create and apply mixins as follows.

type Constructor = new (...args: any[]) => any

const UserWithEmail = <T extends Constructor>(superclass: T) => {
  return class extends superclass {
    public email: string
  }
}

const UserWithPassword = <T extends Constructor>(superclass: T) => {
  return class extends superclass {
    public password: string
  }
}

class BaseModel {}
class User extends UserWithPassword(UserWithEmail(BaseModel)) {}

Mixins are close to perfect way of inherting multiple classes. I recommend reading this article for same.

However, the syntax of applying multiple mixins is kind of ugly, as you have apply mixins over mixins creating a nested hierarchy as shown below.

UserWithAttributes(UserWithAge(UserWithPassword(UserWithEmail(BaseModel))))

The compose method is a small utility to improve the syntax a bit.

import { compose } from '@poppinss/utils'

class User extends compose(
  BaseModel,
  UserWithPassword,
  UserWithEmail,
  UserWithAge,
  UserWithAttributes
) {}

Mixins gotchas

Typescript has an open issue related to the constructor arguments of the mixin class or the base class.

Typescript expects all classes used in the mixin chain to have a constructor with only one argument of ...args: any[]. For example: The following code will work fine at runtime, but the typescript compiler complains about it.

class BaseModel {
  constructor(name: string) {}
}

const UserWithEmail = <T extends typeof BaseModel>(superclass: T) => {
  return class extends superclass {
    // ERROR: A mixin class must have a constructor with a single rest parameter of type 'any[]'.ts(2545)
    public email: string
  }
}

class User extends compose(BaseModel, UserWithEmail) {}

You can work around this by overriding the constructor of the base class.

import { NormalizeConstructor, compose } from '@poppinss/utils'

const UserWithEmail = <T extends NormalizeConstructor<typeof BaseModel>>(superclass: T) => {
  return class extends superclass {
    public email: string
  }
}